Measurement device, processing device, and article manufacturing method

ABSTRACT

A measurement device for measuring a position of an object, includes an illuminator configured to illuminate the object, an image capturing device configured to capture the object illuminated by the illuminator, a calculator configured to obtain the position of the object based on an image obtained by the image capturing device, and a controller configured to control the illuminator and the image capturing unit. The controller outputs timing information indicating a timing determined in accordance with a measurement period that is an overlapping period of an illumination period for causing the illuminator to illuminate the object and an image capturing period for causing the image capturing device to capture the object.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to a measurement device, a processingdevice, and an article manufacturing method.

Description of the Related Art

In recent years, in a step of manufacturing various articles, robotsalternatively perform part of work which has been conventionally done byhumans. For example, there is provided a technique in which a sensorwith an illuminator for generating pattern light and an image capturingdevice is mounted at the distal end of a robot arm, and the position ofa part is estimated based on an image of the part, which has beenobtained by the sensor, while operating the robot arm. In thistechnique, the sensor measure the position and attitude of an article(part) in the local coordinate system (to be referred to as a sensorcoordinate system hereinafter) of the sensor. Then, based on theposition and attitude of the article in the sensor coordinate system andthe position and attitude of the sensor in a global coordinate system(to be referred to as a robot coordinate system hereinafter), it ispossible to measure the position and attitude of the article in therobot coordinate system. Therefore, it is necessary to measure theposition and attitude of the object by the sensor and obtain theposition and attitude of the sensor at the same time.

Each of Japanese Patent No. 5740649 and Japanese Patent Laid-Open No.2012-168135 discloses an image measurement unit for performing acontrast AF search based on a plurality of pieces of image informationcaptured while changing the focus position of an image capturing devicefor capturing an object. The image measurement device disclosed inJapanese Patent No. 5740649 obtains a shift amount between an imagecapturing timing and a focus position obtaining timing, and corrects thefocus position based on the shift amount. The image measurement devicedisclosed in Japanese Patent Laid-Open No. 2012-168135 obtains a focusposition based on a trigger signal output from one of the imagecapturing unit and a position control unit for controlling the focusposition to the other.

With a measurement device for measuring the position of an object basedon an image captured by an image capturing device while illuminating theobject using an illuminator, it is difficult to make an illuminationperiod for causing the illuminator to illuminate the object coincidewith an image capturing period for causing the image capturing device tocapture the object. This is because the responsiveness of theilluminator is different from that of the image capturing device. Whenthe illumination period does not coincide with the image capturingperiod, if the timing of obtaining the position and attitude of thesensor is determined in accordance with one of the periods, asynchronization error occurs between the timing of measuring the objectby the measurement device and the timing of obtaining the position andattitude of the sensor. If there is such synchronization error, it isimpossible to accurately grasp the position of the object in the robotcoordinate system. Japanese Patent No. 5740649 and Japanese PatentLaid-Open No. 2012-168135 do not assume that such synchronization erroroccurs.

SUMMARY OF THE INVENTION

The present invention provides a technique advantageous in reducing asynchronization error.

One of aspects of the present invention provides a measurement devicefor measuring a position of an object, the device comprising: anilluminator configured to illuminate the object; an image capturingdevice configured to capture the object illuminated by the illuminator;a calculator configured to obtain the position of the object based on animage obtained by the image capturing device; and a controllerconfigured to control the illuminator and the image capturing unit,wherein the controller outputs timing information indicating a timingdetermined in accordance with a measurement period that is anoverlapping period of an illumination period for causing the illuminatorto illuminate the object and an image capturing period for causing theimage capturing device to capture the object.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a view showing the arrangement of a processing deviceaccording to an embodiment of the present invention;

FIG. 2 is a block diagram showing the arrangement of a processing deviceaccording to the first embodiment of the present invention;

FIG. 3 is a timing chart showing the operation of the processing deviceaccording to the first embodiment of the present invention;

FIG. 4 is a timing chart showing the operation of a processing deviceaccording to the second embodiment of the present invention;

FIG. 5 is a block diagram showing the arrangement of a processing deviceaccording to the third embodiment of the present invention;

FIG. 6 is a timing chart showing the operation of the processing deviceaccording to the third embodiment of the present invention;

FIG. 7 is a block diagram showing the arrangement of a processing deviceaccording to the fourth embodiment of the present invention;

FIG. 8 is a timing chart showing the operation of the processing deviceaccording to the fourth embodiment of the present invention;

FIG. 9 is a block diagram showing the arrangement of a processing deviceaccording to the fifth embodiment of the present invention;

FIG. 10 is a block diagram showing the arrangement of a processingdevice according to the sixth embodiment of the present invention;

FIG. 11 is a flowchart illustrating one operation of the processingdevice according to the sixth embodiment of the present invention; and

FIG. 12 is a flowchart illustrating another operation of the processingdevice according to the sixth embodiment of the present invention.

DESCRIPTION OF THE EMBODIMENTS

Exemplary embodiments of the present invention will be described belowwith reference to the accompanying drawings.

First Embodiment

FIG. 1 shows the arrangement of a processing device 1 according to anembodiment of the present invention. The processing device 1 recognizesan object 500 and processes it. The processing of the object 500includes, for example, holding (for example, gripping) the object 500,operating (for example, moving or rotating) the object 500, examiningthe object 500, fixing the object 500 to another object, or physicallyor chemically changing the object 500.

The processing device 1 can include a measurement device 700 formeasuring the object 500, and a robot system 800 for processing theobject 500 based on the measurement result of the measurement device700. The measurement device 700 can include a sensor 100, and a computer200 (calculator) for controlling the sensor 100 and processinginformation supplied from the sensor 100. All or some of the functionsof the computer 200 may be incorporated in the sensor 100. Themeasurement of the object 500 can include, for example, measuring theposition of the object 500 or measuring the position and attitude (forexample, orientation, aspect, direction, or tilt) of the object 500. Therobot system 800 can include a robot 300 with a hand 310 for processingthe object 500 and a robot controlling device 400 for controlling therobot 300.

The sensor 100 can be fixed to the hand 310 of the robot 300. Forexample, the robot 300 can operate the hand 310 with respect to aplurality of axes (for example, six axes), and operate the object 500held by the hand 310 with respect to the plurality of axes (six axes).The robot 300 can include an encoder for outputting the control amountsof the plurality of axes of the hand 310. The outputs from the encodercan be used by the robot controlling device 400 to control the hand 310of the robot 300, and used to obtain the position and attitude of thesensor 100.

The measurement device 700 generates a range image of the object 500,and generates coordinate data of a three-dimensional point group on theobject 500 based on the range image. The measurement device 700generates a range image of the object 500, and then performs modelfitting for the range image, thereby detecting the position and attitudeof the object 500. The measurement device 700 generates a grayscaleimage of the object 500, and then performs model fitting for thegrayscale image, thereby detecting the position and attitude of theobject 500. Alternatively, the measurement device 700 generates a rangeimage and grayscale image of the object 500, and detects the positionand attitude of the object 500 based on both the range image and thegrayscale image. Note that if it is unnecessary to detect the attitudeof the object 500, the measurement device 700 can be configured todetect only the position of the object 500.

The range image has, for example, a range value calculated based on theprinciple of triangulation from a captured image obtained by capturing,by the image capturing device of the sensor 100, the object 500illuminated with pattern light from the illuminator of the sensor 100.This method is called an active stereo method. The range image may begenerated from one captured image or generated based on a plurality ofimages captured at the same time. As a method of generating a rangeimage based on one captured image, for example, there is provided amethod of projecting an encoded dot line pattern onto the object 500 andcapturing the object 500.

The sensor 100 can measure the position and attitude of the object 500in the sensor coordinate system based on the image captured by thesensor 100. The robot system 800 can obtain the position and attitude ofthe sensor 100 in the robot coordinate system. Using a matrix Tr_(WS)representing the position and attitude of the object 500 in the sensorcoordinate system and a matrix Tr_(SR) representing the position andattitude of the sensor 100 in the robot coordinate system, a matrixTr_(WR) representing the position and attitude of the object 500 in therobot coordinate system can be calculated by:

Tr _(WR) =Tr _(WS) ×Tr _(SR)  (1)

The sensor 100 can be attached to the distal end of the hand 310 of therobot 300 via a mounter or the like. Therefore, when Tr_(SM) representsthe position and attitude of the sensor 100 in a mounter coordinatesystem and Tr_(MR) represents the position and attitude of the mounterin the robot coordinate system, Tr_(SR) in equation (1) is equal to theproduct of Tr_(SM) and Tr_(MR). That is, the matrix Tr_(WR) representingthe position and attitude of the object 500 in the robot coordinatesystem can be calculated using Tr_(SM) and Tr_(MR), by:

Tr _(WR) =Tr _(WS) ×Tr _(SM) ×Tr _(MR)  (2)

Note that it is possible to know the positional relationship between thesensor 100 and the mounter by calibration performed at the time ofinstallation of the sensor 100. Furthermore, it is possible to know theposition and attitude of the mounter in the robot coordinate systembased on the outputs from the encoder with respect to the respectiveaxes of the robot 300. Calculation of equation (1) or (2) may beexecuted by the computer 200, the sensor 100, or the robot controllingdevice 400.

In the processing device 1, while the robot 300 moves the sensor 100relative to the object 500 by moving the hand 310, the sensor 100captures the object 500. The computer 200 detects the position andattitude of the object 500 in the sensor coordinate system based onimages captured by the sensor 100. On the other hand, the robotcontrolling device 400 obtains the position and attitude of the sensor100 in the robot coordinate system based on the outputs from the encoderof the robot 300. The robot controlling device 400 can calculate theposition and attitude of the object 500 in the robot coordinate systembased on the position and attitude of the sensor 100 in the robotcoordinate system and the position and attitude of the object 500 in thesensor coordinate system. Based on the position and attitude of theobject 500 in the robot coordinate system, the robot controlling device400 controls the robot 300 to pick up the object 500.

In the sensor 100, the illuminator illuminates the object 500 withpattern light, and the image capturing device captures the patternlight. The sensor 100 supplies a captured image to the computer 200, andthe computer 200 can generate a range image by processing the suppliedimage, and calculate the position and attitude of the object 500 basedon the range image. In this method, a measurement period (overlappingperiod) as a period in which an illumination period for causing theilluminator of the sensor 100 to illuminate the object 500 and an imagecapturing period for causing the image capturing device to capture theobject 500 overlap each other contributes to measurement of the object500 in the measurement device 700. That is, in the measurement period,the pattern light entering the photoelectric converter of each pixel ofthe image capturing device is photoelectrically converted, therebyaccumulating the generated charges. If the current time falls within theimage capturing period but outside the illumination period, the patternlight is not photoelectrically converted. If the current time fallswithin the illumination period but outside the image capturing period,the pattern light is not photoelectrically converted.

If the sensor 100 (hand 310) is moving relative to the object 500, theimage capturing device captures an image (this image moves) of thepattern light formed on the image sensing surface of the image capturingdevice during the measurement period. If it is considered that thesensor 100 moves relative to the object 500 at a constant speed, thecomputer 200 calculates the average values of the positions andattitudes of the object 500 during the measurement period. The averagevalues of the positions and attitudes of the object 500 during themeasurement period respectively match the position and attitude of theobject 500 at the midpoint (measurement central time) of the measurementperiod.

The robot controlling device 400 needs to synchronize the timing ofmeasuring, by the measurement device 700, the position and attitude ofthe object 500 in the sensor coordinate system with the timing ofobtaining, by the robot controlling device 400, the position andattitude of the sensor 100 in the robot coordinate system. To do this,the robot controlling device 400 obtains the position and attitude ofthe sensor 100 in the robot coordinate system at the timing of themidpoint of the measurement period.

A measurement error caused by a synchronization error will beexemplified for reference. Assume, for example, that a synchronizationerror between the timing of measuring, by the measurement device 700,the position and attitude of the object 500 in the sensor coordinatesystem and the timing of obtaining, by the robot controlling device 400,the position and attitude of the sensor 100 in the robot coordinatesystem is 1 ms. In this case, if the sensor 100 moves at 1 m/sec bymoving the hand 310 of the robot 300, the position of the object 500converted into the robot coordinate system includes an error of 1 mm.

FIG. 2 shows the arrangement of the processing device 1 according to thefirst embodiment of the present invention. The processing device 1 caninclude the measurement device 700 and the robot system 800. Themeasurement device 700 includes the sensor 100 to obtain the positionand attitude of the object 500 by the active stereo method. Note that ifit is unnecessary to obtain the attitude of the object 500, themeasurement device 700 obtains only the position of the object 500.

The sensor 100 can include an illuminator 10 for illuminating the object500 with pattern light, an image capturing device 20 for capturing theobject 500 illuminated with the pattern light, and a controller 30 forcontrolling the illuminator 10 and the image capturing device 20. Thecontroller 30 outputs a measurement time signal as timing informationindicating a timing determined in accordance with the measurement periodas the overlapping period of an illumination period for causing theilluminator 10 to illuminate the object 500 and an image capturingperiod for causing the image capturing device 20 to capture the object500. The measurement device 700 can include the computer 200(calculator) in addition to the sensor 100. For example, the computer200 generates a range image of the object 500 based on an image suppliedfrom the image capturing device 20 of the sensor 100, and obtains theposition and attitude of the object 500 based on the range image.

The illuminator 10 can include, for example, a light source such as anLED (Light Emitting Diode), a mask for generating pattern light usinglight from the light source, an optical system for projecting thepattern light, and a drive circuit for driving the light source. Themask is a member on which a pattern to be projected onto the object 500is drawn, and can be obtained by, for example, forming a light blockingportion on a glass substrate by chromium plating. Instead of using themask, a DMD (Digital Mirror Device) or liquid crystal panel may beadopted.

The image capturing device 20 can include an image sensor and an imagingoptical system for forming, on the image sensing surface of the imagesensor, an image of the object 500 illuminated with the pattern light.The image sensor can be a CCD image sensor, a CMOS image sensor, or thelike. If a CMOS image sensor is adopted, a global shutter type CMOSimage sensor is desirably used. For a rolling shutter type CMOS imagesensor, an image capturing period (measurement period) is different foreach row, which may cause an error.

The controller 30 can be formed by, for example, a PLD (ProgrammableLogic Device) such as an FPGA (Field Programmable Gate Array), or anASIC (Application Specific Integrated Circuit). The controller 30controls the illuminator 10 and the image capturing device 20 based onmeasurement parameters such as the illumination period and imagecapturing period set by the computer 200.

The measurement parameters such as the illumination period and imagecapturing period may be set by the computer 200 or the robot controllingdevice 400. Alternatively, pre-measurement may be performed prior tomeasurement of the position and attitude of the object 500, and thecontroller 30 may set the illumination period and image capturing periodbased on the result of the pre-measurement. The measurement parameterscan be set in accordance with the reflectance of the object 500. Forexample, if the object 500 is an object having a high reflectance likemetal, the illumination period and image capturing period are set tohave short times. Conversely, if the object 500 is a black object havinga low reflectance, the illumination period and image capturing periodare set to have long times.

The controller 30 can be configured to, for example, start control ofillumination by the illuminator 10 and image capturing by the imagecapturing device 20 in response to a measurement trigger provided fromthe robot controlling device 400. Alternatively, the controller 30 canbe configured to start control illumination by the illuminator 10 andimage capturing by the image capturing device 20 in response to ameasurement trigger provided from the robot controlling device 400 viathe computer 200.

To transmit the measurement parameters from the computer 200 to thecontroller 30, the computer 200 and the controller 30 can be connectedby, for example, an interface such as Ethernet or RS-232C. To provide ameasurement trigger from the robot controlling device 400 to thecontroller 30, the robot controlling device 400 and the controller 30can be connected by, for example, an interface such as a photocoupler.To control the operation of the illuminator 10, the controller 30 andthe illuminator 10 can be connected by, for example, a digital signalline for transmitting an illumination signal, and the controller 30 cancontrol an illumination operation by the illuminator 10 depending onwhether to set the illumination signal to active level. The controller30 and the image capturing device 20 can be connected by, for example, adigital signal line for transmitting an image capturing signal, and thecontroller 30 can control an image capturing operation by the imagecapturing device 20 depending on whether to set the image capturingsignal to active level.

The controller 30 transmits the measurement time signal to theacceleration obtainer 40 to synchronize the timing of measuring, by themeasurement device 700, the position and attitude of the object 500 withthe timing of obtaining, by the robot controlling device 400, theposition and attitude of the sensor 100. The measurement time signal isa signal (timing information) indicating a timing determined inaccordance with the measurement period, more specifically, a signalindicating the timing of the midpoint of the measurement period.

The operation of the processing device 1 according to the firstembodiment will be described below with reference to a timing chartshown in FIG. 3. The measurement trigger (3-a) output from the robotcontrolling device 400 is received by the controller 30 after a lapse ofa transmission delay time τ1 generated in an interface circuit andcommunication path forming the interface between the robot controllingdevice 400 and the controller 30. In response to reception of themeasurement trigger (3-a), the controller 30 starts control formeasurement. More specifically, in response to reception of themeasurement trigger (3-a), the controller 30 sets the illuminationsignal (3-b) to active level for the illumination period, and sets theimage capturing signal (3-c) to active level for the image capturingperiod. As an example, if the cable length of the communication pathforming the interface between the robot controlling device 400 and thesensor 100 is 10 m, the transmission delay time τ1 is dominated by atransmission delay in the interface circuit, and is, for example, aboutseveral tens of μs.

Let T_(L) be the illumination period during which the controller 30causes the illuminator 10 to illuminate the object 500 (the periodduring which the illumination signal is maintained at active level),T_(E) be the image capturing period during which the controller 30causes the image capturing device 20 to capture the object 500 (theperiod during which the image capturing signal is maintained at activelevel), and Δt be the difference between the timing of causing theillumination signal to transit to active level and the timing of causingthe image capturing signal to transit to active level. The computer 200can preset T_(L), T_(E), and Δt as measurement parameters. Asexemplified in FIG. 3, setting the illumination period T_(L) to belonger than the image capturing period T_(E) is useful when, forexample, the time taken to switch the operation state of the imagecapturing device 20 is shorter than the time taken to switch theoperation state of the illuminator 10. For example, Δt can be set inconsideration of the time taken to stabilize illumination lightgenerated by the illuminator 10. The illumination period and the imagecapturing period are not limited to those in the example of FIG. 3, andvarious relationships can be assumed.

As described above, a measurement period T (3-d) as the overlappingperiod of the illumination period T_(L) for causing the illuminator 10to illuminate the object 500 and the image capturing period T_(E)(exposure period) for causing the image capturing device 20 to capturethe object 500 contributes to measurement of the object 500 in themeasurement device 700. Therefore, the controller 30 causes themeasurement time signal (3-f) to transit to the active level at themidpoint (to be referred to as measurement central time hereinafter) ofthe measurement period T. The measurement time signal (3-f) is a pulsesignal in this example. The output of the measurement time signal is anexample of a method of outputting timing information indicating a timingdetermined in accordance with the measurement period T.

The controller 30 can measure a period of T/2 by a timer after the startof the measurement period T (3-e), and output the measurement timesignal (3-f) to the robot controlling device 400 at time when the periodof T/2 elapses. Note that the timer for measuring the period of T/2 canbe configured to, for example, operate by the AND of the illuminationsignal and the image capturing signal. The period of T/2 can becalculated using the illumination period T_(L), the image capturingperiod T_(E), and the difference Δt between the illumination starttiming and the image capturing start timing, by:

$\begin{matrix}{{T\text{/}2} = \left\{ \begin{matrix}{{T_{E}\text{/}2}} & \left( {{{\Delta \; t} \geq 0},{T_{L} \geq {{\Delta \; t} + T_{E}}}} \right) \\{\left( {T_{L} - {\Delta \; t}} \right)\text{/}2} & {\left( {{{\Delta \; t} \geq 0},{T_{L} < {{\Delta \; t} + T_{E}}}} \right)\mspace{25mu}} \\{\left( {T_{E} + {\Delta \; t}} \right)\text{/}2} & {\left( {{{\Delta \; t} < 0},{T_{L} \geq {{\Delta \; t} + T_{E}}}} \right)\mspace{25mu}} \\{{T_{L}\text{/}2}} & \left( {{{\Delta \; t} < 0},{T_{L} < {{\Delta \; t} + T_{E}}}} \right)\end{matrix} \right.} & (3)\end{matrix}$

Note that the value of T/2 changes in accordance with the values ofT_(L), T_(E), and Δt, that is, the start and end timings of theillumination signal and image capturing signal.

T/2 may be calculated by the computer 200 and transmitted to thecontroller 30 together with the measurement parameters, or may becalculated by the controller 30. The controller 30 and the robotcontrolling device 400 can be connected by, for example, a cable and aninterface circuit using a photocoupler. The measurement time signaltransmitted from the controller 30 can be received by the robotcontrolling device 400 after a lapse of a transmission delay time τ2generated in the interface circuit and cable. If, for example, thelength of the cable connecting the sensor 100 and the robot controllingdevice 400 is 10 m, the transmission delay time τ2 is dominated by atransmission delay in the interface circuit, and is, for example, aboutseveral tens of μs. In response to the measurement time signal, therobot controlling device 400 obtains the position and attitude (3-g) ofthe sensor 100 in the robot coordinate system.

As described above, the controller 30 transmits the measurement timesignal to the robot controlling device 400 at measurement central time.This can synchronize measurement of the position and attitude of theobject 500 by the sensor 100 with obtaining of the position and attitudeof the sensor 100 by the robot controlling device 400.

The arrangement of the measurement device 700 will be described withreference to FIG. 2. An image captured by the image capturing device 20is transmitted to the computer 200, and the computer 200 generates arange image based on the image, and detects the position and attitude ofthe object 500 by performing model fitting. To transmit the image fromthe image capturing device 20 to the computer 200, the image capturingdevice 20 and the computer 200 can be connected by, for example,Ethernet. Alternatively, the image capturing device 20 and the computer200 may be connected by another interface such as Camera Link or USB3.0.

The position and attitude of the object 500 calculated by the computer200 are transmitted to the robot controlling device 400 and used asinput data to control driving of the robot 300. To transmit the positionand attitude of the object 500 from the computer 200 to the robotcontrolling device 400, the computer 200 and the robot controllingdevice 400 can be connected by, for example, an interface such asEthernet.

In the processing device 1, the controller 30 controls a measurementoperation based on the set illumination time T_(L), the set imagecapturing period T_(E), and the set difference Δt between illuminationstart time and image capturing start time. Furthermore, the processingdevice 1 outputs the measurement time signal to the robot controllingdevice 400 at measurement central time (the midpoint of the measurementperiod T). This allows the robot controlling device 400 to obtain theposition and attitude of the sensor 100 using measurement central timeas a reference (trigger). Therefore, it is possible to synchronizemeasurement of the position and attitude of the object 500 by the sensor100 with obtaining of the position and attitude of the sensor 100 by therobot controlling device 400.

Second Embodiment

The second embodiment of the present invention will be described below.Matters that are not mentioned as the second embodiment can comply withthe first embodiment. The operation of a processing device 1 accordingto the second embodiment will be described with reference to a timingchart shown in FIG. 4. In the first embodiment, a synchronization erroroccurs between measurement of the position and attitude of the object500 by the sensor 100 and obtaining of the position and attitude of thesensor 100 by the robot controlling device 400 due to the transmissiondelay time τ2 of the measurement time signal. To cope with this, in thesecond embodiment, a controller 30 outputs a measurement time signal toa robot controlling device 400 a transmission delay time τ2 before themidpoint (measurement central time) of a measurement period T.

The transmission delay time τ2 is caused by a signal delay occurring inan interface circuit and cable between the controller 30 and the robotcontrolling device 400. Furthermore, a delay (for example, a delaycaused by software) from when the robot controlling device 400 receivesthe measurement time signal until the robot controlling device 400obtains the position and attitude may be included.

As shown in FIG. 4, the controller 30 outputs the measurement timesignal to the robot controlling device 400 the transmission delay timeτ2 before measurement central time (the midpoint of the measurementperiod T). That is, the controller 30 measures a period of (T/2−τ2) by atimer after the start of the measurement period T (4-e), and outputs themeasurement time signal (4-f) to the robot controlling device 400 attime when the period of (T/2−τ2) elapses. (T/2−τ2) can be calculatedusing an illumination period T_(L), an image capturing period T_(E), adifference Δt between illumination start time and image capturing starttime, and the transmission delay time τ2 (offset value), by:

$\begin{matrix}{{{T\text{/}2} - {\tau \; 2}} = \left\{ \begin{matrix}{{{T_{E}\text{/}2} - {\tau \; 2}}} & \left( {{{\Delta \; t} \geq 0},{T_{L} \geq {{\Delta \; t} + T_{E}}}} \right) \\{{\left( {T_{L} - {\Delta \; t}} \right)\text{/}2} - {\tau \; 2}} & {\left( {{{\Delta \; t} \geq 0},{T_{L} < {{\Delta \; t} + T_{E}}}} \right)\mspace{25mu}} \\{{\left( {T_{E} + {\Delta \; t}} \right)\text{/}2} - {\tau \; 2}} & {\left( {{{\Delta \; t} < 0},{T_{L} \geq {{\Delta \; t} + T_{E}}}} \right)\mspace{25mu}} \\{{{T_{L}\text{/}2} - {\tau \; 2}}} & \left( {{{\Delta \; t} < 0},{T_{L} < {{\Delta \; t} + T_{E}}}} \right)\end{matrix} \right.} & (4)\end{matrix}$

This can be understood as correction of the timing given by equation (3)using the transmission delay time τ2 (offset value).

Note that the transmission delay time τ2 can be determined based ontransmission time at which the controller 30 actually transmits thesignal and reception time at which the robot controlling device 400receives the signal. The transmission delay time τ2 can be preset by thecomputer 200. Furthermore, (T/2−τ2) may be calculated by the computer200 and transmitted to the controller 30 together with measurementparameters, or may be calculated by the controller 30 by transmitting τ2from the computer 200 to the controller 30. According to the secondembodiment, it is possible to reduce a synchronization error, ascompared to the first embodiment.

As described above, the controller 30 generates a measurement timesignal as information indicating a timing obtained by correcting, basedon the preset offset value (τ2), a timing (a timing of T/2 after thestart of the measurement period T) determined in accordance with themeasurement period T.

Third Embodiment

The third embodiment of the present invention will be described below.Matters that are not mentioned as the third embodiment can comply withthe first embodiment. FIG. 5 shows the arrangement of a processingdevice 1 according to the third embodiment of the present invention.FIG. 6 shows the operation of the processing device 1 according to thethird embodiment of the present invention. In the third embodiment, thesensor 100, the computer 200, and the robot controlling device 400according to the first embodiment are replaced with a sensor 100 a, acomputer 200 a, and a robot controlling device 400 a, respectively. Inthe sensor 100 a, the controller 30 according to the first embodiment isreplaced with a controller 30 a.

In the third embodiment, a measurement device 700 transmits, to therobot controlling device 400 a (a robot system 800), a time stamp(digital data indicating time information) as timing informationtogether with information of the position and attitude of an object 500in the sensor coordinate system. Upon receiving the time stamp, therobot controlling device 400 a obtains the position and attitude of thesensor 100 a in the robot coordinate system at time indicated by thetime stamp. The robot controlling device 400 a can obtain the positionand attitude of the sensor 100 a based on, for example, the drivingprofile of a robot 300. Alternatively, the robot controlling device 400a may obtain the position and attitude of the sensor 100 a byinterpolating the output values of an encoder, that have been obtainedat a given sampling interval for driving of a hand 310 of the robot 300.Note that times given by clocks in the measurement device 700 and therobot system 800 desirably coincide with each other. However, themeasurement device 700 and the robot system 800 may be configured torecognize a time shift between them.

The third embodiment can be used in, for example, a status in whichinformation indicating the position and attitude of the object 500 inthe robot coordinate system is not required in real time. For example,the controller 30 a can be configured to start control of illuminationby an illuminator 10 and image capturing by an image capturing device 20in response to a measurement trigger provided from the robot controllingdevice 400 a. Alternatively, the controller 30 a can be configured tostart control of illumination by the illuminator 10 and image capturingby the image capturing device 20 in response to a measurement triggerprovided from the robot controlling device 400 a via the computer 200 a.

Measurement parameters such as an illumination period T_(L) and an imagecapturing period T_(E) may be set by the computer 200 a or the robotcontrolling device 400 a. Alternatively, pre-measurement may beperformed prior to measurement of the position and attitude of theobject 500, and the sensor 100 a may determine and set the measurementparameters based on the result of the pre-measurement. The illuminationperiod T_(L) and the image capturing period T_(E) may be changed inaccordance with an output from a light amount monitor circuit (notshown) during measurement. Alternatively, it may be configured tocontinue at least one of an illumination operation and an imagecapturing operation until specific processing performed duringmeasurement ends. That is, the set illumination period and imagecapturing period may be different from the actual illumination periodand image capturing period. In addition, the illumination period andimage capturing period can be changed by various methods.

In order for the robot controlling device 400 a to obtain the positionand attitude of the sensor 100 a at measurement central time (themidpoint of a measurement period T), the controller 30 a transmits atime stamp as timing information to the computer 200 a. Then, thecomputer 200 a transmits the time stamp to the robot controlling device400 a together with the position and attitude of the object 500 in thesensor coordinate system.

The operation of the processing device 1 according to the thirdembodiment will be described below with reference to a timing chartshown in FIG. 6. The measurement trigger (6-a) output from the robotcontrolling device 400 a is received by the controller 30 a after alapse of a transmission delay time τ1 generated in an interface circuitand communication path forming an interface between the robotcontrolling device 400 a and the controller 30 a. In response toreception of the measurement trigger (6-a), the controller 30 a startscontrol for measurement. More specifically, in response to reception ofthe measurement trigger (6-a), the controller 30 a sets an illuminationsignal (6-b) to active level for the illumination period T_(L), and setsan image capturing signal (6-c) to active level for the image capturingperiod T_(E).

In the example shown in FIG. 6, the illumination period T_(L) endsbefore the end of the image capturing period T_(E). This control isuseful when, in an image readout period (not shown) after the end of theimage capturing period T_(E), noise caused by turning off theilluminator 10 may decrease the S/N ratio of an image output from theimage capturing device 20 or distort a transferred image, thereby posinga problem. In the third embodiment, however, the relationship betweenthe illumination period T_(L) and the image capturing period T_(E) isnot limited to a specific one.

As described above, the measurement period T (6-d) as the overlappingperiod of the illumination period T_(L) for causing the illuminator 10to illuminate the object 500 and the image capturing period T_(E)(exposure period) for causing the image capturing device 20 to capturethe object 500 contributes to measurement of the object 500 in themeasurement device 700. Thus, based on the illumination signal (6-b) andthe image capturing signal (6-c), the controller obtains measurementstart time (6-e) and measurement end time (6-f). This operation can beexecuted based on, for example, the rise and fall of the AND of theillumination signal (6-b) and the image capturing signal (6-c).

After that, the controller 30 a calculates measurement central time (themidpoint of the measurement period T) based on measurement start timeand measurement end time (6-g). The controller 30 a transmitsmeasurement central time as a time stamp to the computer 200 a, and thecomputer 200 a transmits the time stamp to the robot controlling device400 a together with the position and attitude of the object 500 in thesensor coordinate system (6-h). The robot controlling device 400 aobtains the position and attitude of the sensor 100 a in the robotcoordinate system at time indicated by the time stamp (6-i).

Fourth Embodiment

The fourth embodiment of the present invention will be described below.Matters that are not mentioned as the fourth embodiment can comply withthe first embodiment. FIG. 7 shows the arrangement of a processingdevice 1 according to the fourth embodiment of the present invention.FIG. 8 shows the operation of the processing device 1 according to thefourth embodiment of the present invention. In the fourth embodiment,the sensor 100, the computer 200, and the robot controlling device 400according to the first embodiment are replaced with a sensor 100 b, acomputer 200 b, and a robot controlling device 400 b, respectively. Inthe sensor 100 b, the illuminator 10 according to the first embodimentis replaced with a first illuminator 12 and a second illuminator 14, andthe image capturing device 20 according to the first embodiment isreplaced with a first image capturing device 22 and a second imagecapturing device 24. Furthermore, the controller 30 according to thefirst embodiment is replaced with a controller 30 b. The firstilluminator 12 illuminates an object 500 with the first illuminationlight, and the second illuminator 14 illuminates the object 500 with thesecond illumination light.

In the fourth embodiment, the first set of the first illuminator 12 andthe first image capturing device 22 is used to measure the position andattitude of the object 500. Simultaneously with this, the second set ofthe second illuminator 14 and the second image capturing device 24 isused to measure the position and attitude of the object 500. One of thefirst and second sets can be used to capture a grayscale image andmeasure the position and attitude of the object 500 based on thegrayscale image. The other one of the first and second sets can be usedto generate a range image based on an image obtained by illuminationwith pattern light and measure the position and attitude of the object500 based on the range image.

If model fitting is performed for the grayscale image, the estimationaccuracy of the position and attitude in the depth direction of theimage is not so high. On the other hand, if model fitting is performedfor the range image, the estimation accuracy of the position andattitude in the planar direction of the image is not so high. Therefore,a technique of estimating the position and attitude with high accuracyusing both the grayscale image and the range image is useful. However,since light amounts necessary for image capturing operations to obtainthese images are different from each other, measurement times in thefirst and second sets may be different from each other. To cope withthis, in the fourth embodiment, the controller 30 b controls the firstand second sets so that the difference between measurement central timein the first set and that in the second set falls within an allowablerange, preferably, so that measurement central time in the first setcoincides with that in the second set.

The controller 30 b controls the first illuminator 12 of the first setby the first illumination signal, and controls the second illuminator 14of the second set by the second illumination signal. The controller 30 bcontrols the first image capturing device 22 of the first set by thefirst image capturing signal, and controls the second image capturingdevice 24 of the second set by the second image capturing signal. Thecontroller 30 b controls the first and second illumination signals andthe first and second image capturing signals so that measurement centraltime in the first set coincides with that in the second set.Furthermore, the controller 30 b transmits a measurement time signal tothe robot controlling device 400 b at measurement central time tosynchronize measurement of the position and attitude of the object 500by the sensor 100 b with obtaining of the position and attitude of thesensor 100 b by the robot controlling device 400 b. The measurement timesignal is timing information indicating a timing determined inaccordance with the first measurement period determined by the firstillumination signal and the first image capturing signal and the secondmeasurement period determined by the second illumination signal and thesecond image capturing signal.

The operation of the processing device 1 according to the fourthembodiment will be described below with reference to a timing chartshown in FIG. 8. A measurement trigger (8-a) output from the robotcontrolling device 400 b is received by the controller 30 b after alapse of a transmission delay time τ1 generated in an interface circuitand communication path forming an interface between the robotcontrolling device 400 b and the controller 30 b. In response toreception of the measurement trigger (8-a), the controller 30 b startscontrol for measurement. More specifically, in response to reception ofthe measurement trigger (8-a), the controller 30 b sets the firstillumination signal (8-b) to active level for a first illuminationperiod T_(L1), and sets the first image capturing signal (8-c) to activelevel for a first image capturing period T_(E1). In response toreception of the measurement trigger (8-a), the controller 30 b sets thesecond illumination signal (8-h) to active level for a secondillumination period T_(L2), and sets the second image capturing signal(8-i) to active level for a second image capturing period T_(E2).

Let Δt1 be the difference between the timing of causing the firstillumination signal to transit to active level and the timing of causingthe first image capturing signal to transit to active level, and Δt2 bethe difference between the timing of causing the second illuminationsignal to transit to active level and the timing of causing the secondimage capturing signal to transit to active level. The firstillumination period T_(L1), the second illumination period T_(L2), thefirst image capturing period T_(E1), the second image capturing periodT_(E2), the differences Δt1 and Δt2 between illumination start times andimage capturing start times are preset as measurement parameters.

A first measurement period T1 as the overlapping period of the firstillumination period T_(L1) for causing the first illuminator 12 toilluminate the object 500 and the first image capturing period T_(E1)for causing the first image capturing device 22 to capture the object500 contributes to measurement of the object 500 using the first set. Asecond measurement period T2 as the overlapping period of the secondillumination period T_(L2) for causing the second illuminator 14 toilluminate the object 500 and the second image capturing period T_(E2)for causing the second image capturing device 24 to capture the object500 contributes to measurement of the object 500 using the second set.The controller 30 b controls the first and second sets so thatmeasurement central time in the first set coincides with that in thesecond set. In the example shown in FIG. 8, the controller 30 b providesa time difference Δt3 between start time of the first illuminationsignal and that of the second illumination signal. The time differenceΔt3 can be calculated using the differences Δt and Δt2 betweenillumination start times and image capturing start times, and the firstand second measurement periods T1 and T2, by:

Δt3=|(|Δt1|+T1/2)−(|Δt2|+T2/2)|  (5)

wherein T1/2 can be calculated using the illumination period T_(L1), theimage capturing period T_(E1), and the difference Δt1 betweenillumination start time and image capturing start time, by:

$\begin{matrix}{{T\; 1\text{/}2} = \left\{ \begin{matrix}{{T_{E\; 1}\text{/}2}} & \left( {{{\Delta \; t\; 1} \geq 0},{T_{L\; 1} \geq {{\Delta \; t\; 1} + T_{E\; 1}}}} \right) \\{\left( {T_{L\; 1} - {\Delta \; t\; 1}} \right)\text{/}2} & {\left( {{{\Delta \; t\; 1} \geq 0},{T_{L\; 1} < {{\Delta \; t\; 1} + T_{E\; 1}}}} \right)\mspace{25mu}} \\{\left( {T_{E\; 1} + {\Delta \; t\; 1}} \right)\text{/}2} & {\left( {{{\Delta \; t\; 1} < 0},{T_{L\; 1} \geq {{\Delta \; t\; 1} + T_{E\; 1}}}} \right)\mspace{25mu}} \\{{T_{L\; 1}\text{/}2}} & \left( {{{\Delta \; t\; 1} < 0},{T_{L\; 1} < {{\Delta \; t\; 1} + T_{E\; 1}}}} \right)\end{matrix} \right.} & (6)\end{matrix}$

Similarly, T2/2 can be calculated using the illumination period T_(L2),the image capturing period T_(E2), and the difference Δt2 betweenillumination start time and image capturing start time, by:

$\begin{matrix}{{T\; 2\text{/}2} = \left\{ \begin{matrix}{{T_{E\; 2}\text{/}2}} & \left( {{{\Delta \; t\; 1} \geq 0},{T_{L\; 2} \geq {{\Delta \; t\; 1} + T_{E\; 2}}}} \right) \\{\left( {T_{L\; 2} - {\Delta \; t\; 1}} \right)\text{/}2} & {\left( {{{\Delta \; t\; 1} \geq 0},{T_{L\; 2} < {{\Delta \; t\; 1} + T_{E\; 2}}}} \right)\mspace{25mu}} \\{\left( {T_{E\; 2} + {\Delta \; t\; 1}} \right)\text{/}2} & {\left( {{{\Delta \; t\; 1} < 0},{T_{L\; 2} \geq {{\Delta \; t\; 1} + T_{E\; 2}}}} \right)\mspace{25mu}} \\{{T_{L\; 2}\text{/}2}} & \left( {{{\Delta \; t\; 1} < 0},{T_{L\; 2} < {{\Delta \; t\; 1} + T_{E\; 2}}}} \right)\end{matrix} \right.} & (7)\end{matrix}$

Δt3, T1/2, and T2/2 may be calculated by the computer 200 b andtransmitted to the controller 30 b together with the measurementparameters, or may be calculated by the controller 30 b. The controller30 b measures a period of T1/2 by a timer after the start of themeasurement period (8-e), and outputs the measurement time signal to therobot controlling device 400 b at time when the period of T1/2 elapses.Note that the timer for measuring the period of T1/2 can be configuredto, for example, operate by the AND of the first illumination signal andthe first image capturing signal. The measurement time signal can bereceived by the robot controlling device 400 b after a lapse of atransmission delay time τ2 generated in the interface circuit and cable.In response to the measurement time signal, the robot controlling device400 b obtains the position and attitude (8-g) of the sensor 100 b in therobot coordinate system. Note that according to the method described inthe second embodiment, a synchronization error occurring due to thetransmission delay time τ2 may be reduced.

As described above, by determining an operation timing according toequation (5), measurement central time in the first set can be made tocoincide with that in the second set. Furthermore, by transmitting themeasurement time signal to the robot controlling device 400 b atmeasurement central time, it is possible to synchronize measurement ofthe position and attitude of the object 500 by the sensor 100 b withobtaining of the position and attitude of the sensor 100 b by the robotcontrolling device 400 b.

Images captured by the first illuminator 12 and the second illuminator14 are transmitted to the computer 200 b, and the computer 200 bcalculates the position and attitude of the object 500 in the sensorcoordinate system based on these images. In the fourth embodiment, tosynchronize measurement of the position and attitude of the object 500by the sensor 100 b with obtaining of the position and attitude of thesensor 100 b by the robot controlling device 400 b, the controller 30 boutputs the measurement time signal to the robot controlling device 400b. However, as in the third embodiment, the controller 30 b may output atime stamp to the robot controlling device 400 b.

In the fourth embodiment, the two sets of the illuminators and imagecapturing devices are provided. However, three or more sets may beprovided. In this case as well, control is performed so that thedifference between measurement central times in the three or more setsfalls within an allowable range, preferably, so that measurement centraltimes in the three or more sets coincide with each other. Furthermore,timing information (measurement time information) indicating a timingdetermined in accordance with measurement central time can be output.

Fifth Embodiment

The fifth embodiment of the present invention will be described below.Matters that are not mentioned as the fifth embodiment can comply withthe fourth embodiment. FIG. 9 shows the arrangement of a processingdevice 1 according to the fifth embodiment of the present invention. Inthe fifth embodiment, the sensor 100 b, the computer 200 b, and therobot controlling device 400 b according to the fourth embodiment arereplaced with a sensor 100 c, a computer 200 c, and a robot controllingdevice 400 c, respectively. In the sensor 100 c, the first illuminator12 and the second illuminator 14 according to the fourth embodiment arereplaced with a first illuminator 12 c and a second illuminator 14 c,respectively. The first illuminator 12 c illuminates an object 500 withthe first illumination light, and the second illuminator 14 cilluminates the object 500 with the second illumination light.

In the fifth embodiment, the first set of the first illuminator 12 c anda first image capturing device 22 c is used to measure the position andattitude of the object 500. Simultaneously with this, the second set ofthe second illuminator 14 c and a second image capturing device 24 c isused to measure the position and attitude of the object 500. One of thefirst and second sets can be used to capture a grayscale image andmeasure the position and attitude of the object 500 based on thegrayscale image. The other one of the first and second sets can be usedto generate a range image based on an image obtained by illuminationwith pattern light and measure the position and attitude of the object500 based on the range image.

In the fourth embodiment, the timings of the first and secondillumination signals and the first and second image capturing signalsare controlled so that the difference between measurement central timein the first set and that in the second set falls within an allowablerange, preferably, so that measurement central time in the first setcoincides with that in the second set. On the other hand, in the fifthembodiment, the measurement period of the first set is made to match themeasurement period of the second set, thereby making measurement centraltime in the first set coincide with that in the second set. Inaccordance with the measurement period, the intensity of the firstillumination light and that of the second illumination light aredetermined. The intensity of the first illumination light is controlledby the first intensity command value sent to the first illuminator 12 cby a controller 30 c, and the intensity of the second illumination lightis controlled by the second intensity command value sent to the secondilluminator 14 c by the controller 30 c.

The controller 30 c controls the first illuminator 12 of the first setand the second illuminator 14 of the second set by a common illuminationsignal. Furthermore, the controller 30 c controls the first imagecapturing device 22 of the first set and the second image capturingdevice 24 of the second set by a common image capturing signal.

As an example, consider a case in which, as recommended conditions, arecommended illumination period by the first illuminator 12 c is 20 ms,a recommended illumination period by the second illuminator 14 c is 15ms, a recommended image capturing period by the first image capturingdevice 22 is 15 ms, and a recommended image capturing period by thesecond image capturing device 24 is 10 ms. Assume that each ofrecommended intensity command values for the first illuminator 12 c andsecond illuminator 14 c is 50%. Assume also that a recommendedmeasurement period by the first illuminator 12 c and the first imagecapturing device 22 and that by the second illuminator 14 c and thesecond image capturing device 24 are 15 ms and 10 ms, respectively.

In this example, consider a case in which under a policy of setting ashorter common illumination period and a shorter common image capturingperiod, the common illumination period, the common image capturingperiod, and the common measurement time are set to 15 ms, 10 ms, and 10ms, respectively. In this case, the light amount of an image obtained bythe first image capturing device 22 of the first set decreases to ⅔ of alight amount obtained under the recommended conditions, and the lightamount of an image obtained by the second image capturing device 24 ofthe second set is equal to the light amount obtained under therecommended conditions. To compensate for the decrease amount, the firstintensity command value for the first illuminator 12 c is changed fromthe recommended intensity command value of 50% to 75%. Furthermore, asfor the second intensity command value for the second illuminator 14 c,the recommended intensity command value of 50% is maintained. This canmake measurement central time in the first set coincide with that in thesecond set.

To synchronize measurement of the position and attitude of the object500 by the sensor 100 c with obtaining of the position and attitude ofthe sensor 100 c by the robot controlling device 400 c, the controller30 c transmits the measurement time signal to the robot controllingdevice 400 c at measurement central time. Instead of outputting themeasurement time signal from the controller 30 c to the robotcontrolling device 400 c, the controller 30 c may output a time stamp tothe robot controlling device 400 c, as in the third embodiment.

In the fifth embodiment, the two sets of the illuminators and imagecapturing devices are provided. However, three or more sets may beprovided. In this case as well, measurement central times in the threeor more sets can be made to coincide with each other.

Sixth Embodiment

The sixth embodiment of the present invention will be described below.Matters that are not mentioned as the sixth embodiment can comply withthe first embodiment. FIG. 10 shows the arrangement of a processingdevice 1 according to the sixth embodiment of the present invention. Inthe sixth embodiment, the sensor 100 according to the first embodimentis replaced with a sensor 100 d. In the sensor 100 d, the controller 30according to the first embodiment is replaced with a controller 30 d,and an acceleration obtainer 40 is added. The acceleration obtainer 40obtains the acceleration of the sensor 100 d (an illuminator 10 and animage capturing device 20).

In the first to fifth embodiments, the position and attitude of theobject 500 are measured on the assumption that the hand 310 of the robot300 moves at a constant speed. However, the hand 310 of the robot 300may make an acceleration motion. The acceleration motion includes anacceleration motion that increases the speed and an acceleration motion(deceleration motion) that decreases the speed. In the sixth embodiment,even if a hand 310 (the sensor 100 d) of a robot 300 makes anacceleration motion, a synchronization error occurring betweenmeasurement of the position and attitude of an object 500 and obtainingof the position and attitude of the sensor 100 d is reduced. In thesixth embodiment, the controller 30 d outputs timing information(measurement central time) in accordance with the acceleration of thesensor 100 d (illuminator 10 and image capturing device 20) in additionto a measurement period. The acceleration of the sensor 100 d isobtained by the acceleration obtainer 40.

If the absolute value of the acceleration output from the accelerationobtainer 40 is smaller than a threshold, that is, it can be consideredthat the sensor 100 d makes a uniform motion, the controller 30 d canoutput measurement central time as timing information. However, if theabsolute value of the acceleration output from the acceleration obtainer40 is larger than the threshold, that is, it cannot be considered thatthe sensor 100 d makes a uniform motion, a synchronization error mayoccur by obtaining the position and attitude of the sensor 100 d atmeasurement central time.

For example, in the active stereo method in which an object isilluminated with pattern light, after performing processing such ascenter-of-gravity detection or peak detection for the luminance value ofa pattern image in a captured image, a range image can be calculated.Therefore, if image capturing is performed in a state in which thesensor 100 d makes an acceleration motion, the pattern image in thecaptured image is distorted. Thus, the detected center of gravity orpeak position becomes close to a position on the measurement startposition side with respect to the midpoint of the measurement period atthe time of acceleration to increase the speed, and becomes close to aposition on the measurement end position side at the time ofacceleration (deceleration) to decrease speed.

In consideration of this, to reduce a synchronization error caused bythe acceleration motion of the sensor 100 d, a method of measuring theposition and attitude of the object 500 only when the absolute value ofthe acceleration is smaller than the threshold is useful. Alternatively,to reduce a synchronization error caused by the acceleration motion ofthe sensor 100 d, a method of adjusting the timing of obtaining theposition and attitude of the sensor 100 d in accordance with theacceleration is useful.

An operation of measuring the position and attitude of the object 500only when the absolute value of the acceleration is smaller than thethreshold will be described with reference to FIG. 11. In step S101, thecontroller 30 d obtains the acceleration from the acceleration obtainer40. Δt this time, the acceleration obtainer 40 obtains an accelerationprofile held by a robot controlling device 400, and obtains theacceleration of the sensor 100 d (hand 310) based on the accelerationprofile. The acceleration obtainer 40 may obtain, from the robotcontrolling device 400, an acceleration detected by an accelerationsensor provided in the hand 310. Alternatively, the accelerationobtainer 40 can include an acceleration sensor to obtain theacceleration of the sensor 100 d (hand 310) based on an output from theacceleration sensor.

In step S102, the controller 30 d determines whether the absolute valueof the acceleration obtained in step S101 is smaller than a presetthreshold. If the absolute value is smaller than the threshold, theprocess advances to step S103; otherwise, the process returns to stepS101. Steps S101 and S102 are understood as processing that advances tostep S103 after the absolute value of the acceleration becomes smallerthan threshold. In step S103, the controller 30 d determines whether ameasurement trigger has been received from the robot controlling device400. If the measurement trigger has been received, the process advancesto step S104; otherwise, the process returns to step S101. The thresholdcan be determined based on an allowable synchronization error bymeasuring, in advance, the position and attitude of the object 500 usingthe acceleration as a parameter.

In step S104, the controller 30 d starts control for measurement. Theprocessing for measurement can comply with, for example, the firstembodiment. For example, the controller 30 d sets an illumination signalto active level, and also sets an image capturing signal to activelevel. In step S105, the controller 30 d outputs a measurement timesignal (timing signal) at measurement central time.

Step S103 may be executed before step S101. In this case, if ameasurement trigger is received, steps S101 and S102 are executed. If itis determined in step S102 that the absolute value of the accelerationis smaller than the threshold, step S104 is executed.

An operation of adjusting the timing of obtaining the position andattitude of the sensor 100 d in accordance with the acceleration will bedescribed with reference to FIG. 12. In step S201, the controller 30 dobtains the acceleration from the acceleration obtainer 40. In stepS202, the controller 30 d determines whether a measurement trigger hasbeen received from the robot controlling device 400. If the measurementtrigger has been received, the process advances to step S203; otherwise,the process returns to step S201. Note that step S202 may be executedbefore step S201. In this case, if a measurement trigger is received,step S201 can be executed.

In step S203, the controller 30 d starts control for measurement.Processing for measurement can comply with, for example, the firstembodiment. For example, the controller 30 d sets the illuminationsignal to active level, and also sets the image capturing signal toactive level.

In step S204, it is determined whether the absolute value of theacceleration obtained in step S201 is smaller than a preset threshold.If the absolute value is smaller than the threshold, the processadvances to step S205; otherwise, the process advances to step S206. Ifthe absolute value of the acceleration is smaller than the presetthreshold, the controller 30 d outputs, in step S205, the measurementtime signal (timing signal) at measurement central time.

If the absolute value of the acceleration is larger than the presetthreshold, the controller 30 d determines in step S206 whether theacceleration decreases the speed. If the acceleration decreases thespeed, the process advances to step S207. If the acceleration increasesthe speed, the process advances to step S208.

If the acceleration decreases the speed, the controller 30 d outputs, instep S207, as timing information, information indicating a timing (forexample, the end timing of the measurement period) after the midpoint ofthe measurement period. More specifically, the controller 30 d can beconfigured to output a measurement time signal at a timing after themidpoint of the measurement period.

On the other hand, if the acceleration increases the speed, thecontroller 30 d outputs, in step S208, as timing information,information indicating a timing (for example, the start timing of themeasurement period) before the midpoint of the measurement period. Morespecifically, the controller 30 d can be configured to output themeasurement time signal at a timing before the midpoint of themeasurement period.

The threshold used for determination in step S204 can be set to anacceleration that shifts the center of gravity or peak of the patternimage in the captured image from the center of the pattern image widthby ¼ of the pattern image width, by measuring, in advance, the positionand attitude of the object 500 using the acceleration as a parameter.

In the above example, the output timing (timing information) of themeasurement time signal is determined in accordance with each of a casein which the acceleration has an absolute value smaller than thethreshold, a case in which the acceleration decreases the speed, and acase in which the acceleration increases the speed. To more finelycontrol the output timing of the measurement time signal, theacceleration may be divided into a plurality of ranges, and the outputtiming (timing information) of the measurement time signal may bedetermined in accordance with which of the plurality of ranges thedetected acceleration belongs to. A division number can be determined inconsideration of an allowable synchronization error, the degree ofinfluence of the acceleration on the result of measuring the positionand attitude (that is, an algorithm for measuring the position andattitude), or the like.

The timing of measuring the position and attitude of the object 500 maybe adjusted based on a speed obtained by integrating the acceleration,instead of the acceleration. Alternatively, the timing of obtaining theposition and attitude of the sensor 100 d may be adjusted based on thespeed and acceleration. For example, if the speed and accelerationexceeding the field of view of the sensor 100 d are detected during themeasurement period, an error may occur in measurement of the positionand attitude of the object 500. To cope with this, if such speed andacceleration are detected, it may be configured not to measure theposition and attitude of the object 500. Alternatively, if such speedand acceleration are detected, it may be configured to adjust the timingof obtaining the position and attitude of the sensor 100 d inconsideration of the error.

In the above embodiment, the measurement device 700 is fixed to the hand310 of the robot 300, and moves together with the hand 310. The presentinvention is applicable to a case in which the relative position (andrelative acceleration) between the measurement device 700 and the object500 changes when the object 500 moves. In this case, for example, amoving mechanism for moving the object 500 can be provided in place ofthe robot system 800. The moving mechanism can include, for example, amovable member such as a stage that moves together with the object 500,and a movable member controlling device that controls the movement ofthe movable member. In this case, the controller 30 d can be configuredto output timing information in accordance with information of theacceleration of the object 500 (movable member) that can be providedfrom the movable member controlling device, in addition to themeasurement period.

[Supplement]

It is also possible to decrease communication lines by providing a hubmounted in the sensor, such as an Ethernet@ switching hub, between thesensor and the computer, and time-divisionally transmitting/receivingmeasurement parameters and images.

[Article Manufacturing Method]

An object 500 shown in FIG. 1 can be a part for manufacturing (working)an article. The part is processed (for example, worked, assembled, held,or moved) by a robot 300. The robot 300 is controlled by a robotcontrolling device 400. The robot controlling device 400 receivesinformation of the position and attitude of the object 500 from ameasurement device 700, and controls the operation of the robot 300based on the information. A processing device 1 can be formed as amanufacturing device that measures the position (or the position andattitude) of the object 500 by the measurement device 700, processes theobject 500 by a robot system 800 based on the measurement result, andmanufactures an article including the object 500.

An article manufacturing method performed by the processing device 1manufactures an article while operating, based on the measurement resultof the measurement device 700, the robot 300 to which the measurementdevice 700 is attached. The article manufacturing method can include ameasurement step of measuring the position (or the position andattitude) of the object 500 (part) by the measurement device 700, and acontrol step of controlling the robot 300 based on the position (or theposition and attitude) obtained in the measurement step. Control of therobot 300 in the control step includes control for processing an object.This processing can include at least one of, for example, working,cutting, conveyance, assembly, examination, and selection. The articlemanufacturing method according to this embodiment is superior to aconventional method in at least one of the performance, quality,productivity, and production cost of the article.

Other Embodiments

Embodiment(s) of the present invention can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment(s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s). Thecomputer may comprise one or more processors (e.g., central processingunit (CPU), micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM), a read only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2017-045249, filed Mar. 9, 2017, which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. A measurement device for measuring a position ofan object, the device comprising: an illuminator configured toilluminate the object; an image capturing device configured to capturethe object illuminated by the illuminator; a calculator configured toobtain the position of the object based on an image obtained by theimage capturing device; and a controller configured to control theilluminator and the image capturing unit, wherein the controller outputstiming information indicating a timing determined in accordance with ameasurement period that is an overlapping period of an illuminationperiod for causing the illuminator to illuminate the object and an imagecapturing period for causing the image capturing device to capture theobject.
 2. The device according to claim 1, wherein the controller sendsan illumination signal indicating the illumination period to theilluminator, and sends an image capturing signal indicating the imagecapturing period to the image capturing device, and wherein thecontroller generates the timing information in accordance with theillumination signal and the image capturing signal.
 3. The deviceaccording to claim 2, wherein the illumination period is different fromthe image capturing period.
 4. The device according to claim 1, whereinthe controller outputs the timing information in accordance with anacceleration of the illuminator and the image capturing device, inaddition to the measurement period.
 5. The device according to claim 4,wherein the controller obtains the acceleration based on informationobtained from a robot controlling device configured to control a robotto which the measurement device is attached.
 6. The device according toclaim 4, further comprising an acceleration sensor, wherein thecontroller obtains the acceleration based on an output from theacceleration sensor.
 7. The device according to claim 1, wherein thecontroller outputs the timing information in accordance with anacceleration of the object, in addition to the measurement period. 8.The device according to claim 7, wherein the controller obtains theacceleration based on information provided from a movable membercontrolling device configured to control movement of the object.
 9. Thedevice according to claim 4, wherein in a case where the acceleration isfor increasing speed, the controller outputs, as the timing information,information indicating a timing before a midpoint of the measurementperiod, and in a case where the acceleration is for decreasing, thecontroller outputs, as the timing information, information indicating atiming after the midpoint of the measurement period.
 10. The deviceaccording to claim 4, wherein in a case where an absolute value of theacceleration is smaller than a threshold, the controller outputs, as thetiming information, information indicating a timing corresponding to amidpoint of the measurement period.
 11. The device according to claim 4,wherein in a case where an absolute value of the acceleration is smallerthan a threshold, the controller operates the illuminator and the imagecapturing device so that the object is captured.
 12. The deviceaccording to claim 11, wherein in a case where the absolute value of theacceleration is larger than the threshold, the controller does notoperate the illuminator and the image capturing device.
 13. The deviceaccording to claim 4, wherein in a case where an absolute value of theacceleration is smaller than a threshold, the controller operates theilluminator and the image capturing device so that the object iscaptured in response to reception of a measurement trigger.
 14. Thedevice according to claim 1, wherein the timing information is a pulsesignal.
 15. The device according to claim 1, wherein the timinginformation includes digital data indicating time information.
 16. Thedevice according to claim 1, wherein the controller generates, as thetiming information, information indicating a timing obtained bycorrecting, based on a preset offset value, a timing determined inaccordance with at least the measurement period.
 17. The deviceaccording to claim 1, wherein a first illuminator configured toilluminate the object with first illumination light and a secondilluminator configured to illuminate the object with second illuminationlight are provided as the illuminator, and a first image capturingdevice configured to capture the object illuminated with the firstillumination light and a second image capturing device configured tocapture the object illuminated with the second illumination light areprovided as the image capturing device, and a measurement period in afirst set of the first illuminator and the first image capturing deviceand a measurement period in a second set of the second illuminator andthe second image capturing device are determined so that a differencebetween a position of the object obtained using the first set and aposition of the object obtained using the second set falls within anallowable range.
 18. The device according to claim 1, wherein a firstilluminator configured to illuminate the object with first illuminationlight and a second illuminator configured to illuminate the object withsecond illumination light are provided as the illuminator, and a firstimage capturing device configured to capture the object illuminated withthe first illumination light and a second image capturing deviceconfigured to capture the object illuminated with the secondillumination light are provided as the image capturing device, and ameasurement period in a first set of the first illuminator and the firstimage capturing device, a measurement period in a second set of thesecond illuminator and the second image capturing device, an intensityof the first illumination light, and an intensity of the secondillumination light are determined so that a difference between aposition of the object obtained using the first set and a position ofthe object obtained using the second set falls within an allowablerange.
 19. A processing device comprising: a robot with a hand; a robotcontroller configured to control the robot; and a measurement devicedefined by claim 1, wherein an illuminator and an image capturing deviceof the measurement device are attached to the hand, and the robotcontroller controls the robot based on a measurement result of themeasurement device.
 20. An article manufacturing method of manufacturingan article while operating, based on a measurement result of ameasurement device defined by claim 1, a robot to which the measurementdevice is attached, the method comprising: measuring a position of anobject by the measurement device; and controlling the robot based on theposition obtained in the measuring.